Convexity and Characterization of Optimal Policies in a Dynamic Routing Problem'

An infinite horizon, expected average cost, dynamic routing problem is formulated for a simple failure-prone queueing system, modelled as a continuous tithe, continuous state controlled stochastic process. We prove that the optimal average cost is independent of the initial state and that the cost-to-go functions of dynamic programming are convex. These results, together with a set of optimality conditions, lead to the conclusion that optimal 'olicies are switching policies, characterized by a set of switching curves (or regions), each curve corresponding to a particular state of the nodes (servers).